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ABSTRACT 


Conducting  training  in  a  new  or  unfamiliar  environment  requires  a  certain  amount 
of  time  to  acquire  the  necessary  spatial  orientation  and  familiarity  to  that  environment’s 
physical  layout.  This  thesis  explores  the  effects  of  exposing  individuals  to  a  PC-based 
virtual  replication  of  a  small-scale  real  world  training  environment  to  determine  if  such 
exposure  has  any  effect  on  an  individual’s  ability  to  acquire  the  necessary  spatial 
orientation  and  familiarity  of  the  real  world  environment.  In  this  thesis  individual  spatial 
orientation  and  familiarity  to  the  layout  of  the  real  world  environment  will  be  measured 
using  a  set  of  retrieval  tasks  conducted  in  the  real  environment  and  by  development  of  a 
sketched  map  of  that  environment.  Establishing  a  link  between  an  individual’s  ability  to 
gain  an  acceptable  level  of  spatial  orientation  and  familiarity  with  a  real  world 
environment  by  first  exposing  them  to  a  PC-based  virtual  replication  of  that  environment 
is  vital  to  the  future  of  video  game  development  and  virtual  simulation  technologies  used 
for  training  in  the  military. 
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I.  INTRODUCTION 


A.  PROBLEM  STATEMENT 

The  focus  of  this  research  is  to  explore  and  measure,  using  a  PC-based  virtual 
environment,  participants’  level  of  spatial  orientation  and  familiarity  to  a  new  real  world 
environment  by  allowing  them  to  perfonn  a  set  of  specific  tasks  within  that  real  world 
environment.  If  data  reveals  there  is  a  correlation  between  pre-exposure  to  a  real  world 
environment,  it  may  be  possible  to  use  a  PC-based  virtual  replication  of  that  environment 
to  acquire  a  level  of  spatial  orientation  and  familiarity  with  that  environment  adequate 
enough  to  conduct  an  evaluated  task.  If  so,  using  such  a  virtual  environment  may  reduce 
the  time  in  the  actual  location  to  raise  a  service  member’s  knowledge  of  that  location  to  a 
required  level.  In  addition,  future  video  game-based  simulators  and  trainers  could 
develop  environments  with  the  knowledge  that  not  only  are  they  able  to  train  a  specific 
set  of  skills,  but  also  the  environment  in  which  the  game  is  scoped.  The  focus  of  this 
research  is  to  explore  and  measure,  using  a  PC-based  virtual  environment,  the 
effectiveness  of  acquiring  a  adequate  level  of  spatial  orientation  and  familiarity  to  a  new 
real  world  environment  by  allowing  participants  to  perfonn  a  set  of  specific  tasks  within 
that  real  world  environment.  If  data  reveals  there  is  a  correlation  between  pre-exposure  to 
a  real  world  environment,  using  a  PC-based  virtual  replication  of  that  environment,  and 
acquisition  of  a  level  of  spatial  orientation  and  familiarity  with  that  environment  adequate 
enough  to  conduct  evaluated  performance  task,  viable  man  hours  used  to  get  a  service 
member  up  to  speed  on  an  environment’s  physical  layout  could  be  greatly  reduced.  In 
addition,  future  video  game-based  simulators  and  trainers  could  develop  environments 
with  the  knowledge  that  not  only  are  they  able  to  train  a  specific  set  of  skills,  but  also  the 
environment  in  which  the  game  is  scoped. 

B.  RESEARCH  QUESTION 

Can  pre-exposure  to  a  PC-based  virtual  replication  or  model  of  an  actual  training 
environment  provide  participants  with  an  adequate  level  of  spatial  knowledge  to  perfonn 
similarly  on  specific  task  as  participants  who  were  pre-exposed  to  the  real  world  training 
environment? 


1 


C.  DISCUSSION  AND  MOTIVATION 

The  most  effective  method  of  becoming  spatial  orientated  or  familiar  with  the 
layout  of  a  new  or  unfamiliar  environment  is  to  physically  step  foot  into  that 
environment,  touch  the  walls,  smell  the  air,  and  listen  to  the  sounds  that  may  or  may  not 
be  there.  However,  this  is  often  impractical  or  impossible,  especially  in  the  military 
context.  Unfortunately,  this  often  has  significant  consequences  for  the  military. 

Consider,  for  example,  a  newly  enlisted  Sailor  who  reports  to  his  first  shipboard 
tour.  To  say  that  this  environment  is  new  and  unfamiliar  is  a  dramatic  understatement.  It 
will  take  this  Sailor  several  days,  if  not  weeks,  to  properly  acclimate  him  to  the  physical 
layout  of  the  new  shipboard  environment.  During  this  time,  the  sailor  is  of  little  use  as  a 
member  of  the  crew.  Now  give  that  same  Sailor  a  technology  that  pre-exposes  him  to  the 
physical  layout  of  the  interior  shipboard  environment;  in  this  case,  a  PC-based  virtual 
replication  or  model  of  the  interior  spaces  onboard  the  ship  before  he  ever  arrives  for 
duty.  Using  this  virtual  model  of  the  ship,  he  is  able  to  explore  the  many  spaces  within 
the  interior  confines  in  the  hull  of  the  ship.  Upon  completion  of  his  exposure  to  the 
shipboard  environment  in  the  virtual  model  and  finally  reporting  aboard,  he  is  already 
days  or  even  weeks  ahead  of  what  he  might  have  been,  in  tenns  of  spatial  familiarity, 
with  out  the  pre-exposure  to  the  virtual  shipboard  environment.  The  time  it  will  take  for 
the  Sailor  to  properly  acclimate  to  the  new  environment  is  significantly  reduced  and  the 
Sailor  more  quickly  becomes  a  critical  member  of  the  ship’s  crew  and  emergency  teams. 

The  previous  scenario  serves  to  illustrate  a  need  for  providing  a  safe  and 
inexpensive  way  for  not  only  Sailors,  but  all  who  report  to  a  new  or  unfamiliar 
environment  and  are  expected  to  perform  some  type  of  task  requiring  spatial  knowledge 
of  that  environment.  Whether  the  task  is  conducting  training,  emergency  response,  search 
and  rescue,  or  even  nonnal  day  to  day  operations,  a  basic  level  of  spatial  orientation  and 
familiarity  may  be  greatly  enhanced  from  pre-exposure  to  that  environment  leveraging 
PC-based  virtual  environment  technologies.  In  Addition  the  use  of  a  PC-based  virtual 
replication  of  a  real  world  environment  may  also  be  ideal  for  refreshing  an  individual’s 
spatial  orientation  or  familiarity  to  an  enviromnent  that  they  may  have  been  removed 
from  for  some  period  of  time  or  have  only  seen  a  few  times  before. 
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The  motivation  for  this  thesis  is  in  determining  if  pre-exposure  to  a  PC-based 
virtual  environment  affects  an  individual’s  ability  to  acquire  a  similar,  if  not  higher,  level 
of  spatial  orientation  and  familiarity  than  an  individual  pre-exposed  to  the  real  world 
environment.  Toward  that  end,  the  findings  from  this  study  will  set  a  benchmark  for 
future  PC-based  video  gamed  trainers  and  simulators  in  the  military  in  which  knowledge 
of  a  specific  environment’s  physical  layout  is  essential  for  successful  task  completion. 

If  results  indicate  that  pre-exposure  to  a  PC-based  virtual  replication  of  a  real 
world  small-scale  environment  reduces  the  time  for  an  individual  to  a  complete  task  (i.e. 
find  one’s  way  around  the  environment  quicker)  or  at  least  be  familiar  enough  with  the 
physical  layout  of  the  real  environment  to  produce  an  adequate  sketch  of  that 
environment,  time  spent  on  acclimating  military  personnel  to  the  spatial  aspects  of  a  new 
space  may  be  effectively  reduced. 

The  target  population  for  the  experiment  covers  a  wide  range  of  domains,  from 
the  Anny’s  dismounted  infantry  Soldier  preparing  to  enter  an  unfamiliar  building  on  a 
search  and  rescue  mission,  to  the  Navy’s  newest  Sailor  reporting  aboard  his  tour  aboard  a 
ship.  If  PC-based  virtual  environments  were  available  and  proved  to  be  successful  in 
training  spatial  orientation  and  familiarity,  man  hours  and  elaborate  simulation  cost  could 
be  significantly  reduced. 

1.  Relevance  of  Research 

The  relevance  of  this  research  will  be  to  provide  basic  criteria  for  the  US  Navy 
and,  in  general,  the  Department  of  Defense  to  develop  PC-based  virtual  gaming 
technologies  that  can  provide  an  individual  with  a  requisite  level  of  spatial  knowledge  to 
a  real  world  environmental.  The  need  for  a  particular  military  unit  to  become  acclimated 
to  a  new  environment’s  physical  layout  before  ever  arriving  at  that  environment  is 
essential.  Reducing  the  training  time  to  properly  acclimate  a  trainee  to  a  new  training 
environment,  allowing  more  time  to  be  spent  on  specific  skill  sets  necessary  for  task 
accomplishment,  is  also  a  relevant  focus  of  this  research.  The  Navy  could  expose 
shipboard  firefighters  and  emergency  response  teams  to  an  inexpensive  PC-based  virtual 
replication  of  a  shipboard  environment  to  facilitate  training  in  the  real  world 
environment.  Army  dismounted  infantry  Soldiers  could  explore  a  virtual  replication  of  a 
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building  before  stonning  the  actual  building,  knowing  that  once  inside,  their  level  of 
spatial  orientation  and  familiarity  with  the  interior  layout  of  that  building  will  enhance 
their  ability  to  complete  their  mission. 

D.  THESIS  ORGANIZATION 

The  focal  point  of  this  thesis  will  be  to  conduct  a  research  experiment  focusing  on 
using  a  PC-based  virtual  representation  of  an  actual  US  Navy  real-world  training 
environment  to  pre-expose  participants  to  the  physical  layout  of  the  actual  training 
facility.  After  a  predetennined  time  of  exposure  to  the  virtual  training  environment, 
participants  will  complete  selected  perfonnance  measures  to  qualify  their  perfonnance  in 
the  real  world  environment.  The  perfonnance  measures  of  each  participant  exposed  to  the 
PC-based  virtual  environment  will  be  compared  to  participants  who  were  pre-exposed  to 
the  real  training  environment.  The  experiment  involves  using  two  randomly  selected 
groups  of  U.S.  Navy  enlisted  personnel  with  no  prior  knowledge  of  the  actual  training 
environment.  The  control  group  will  be  pre-exposed  to  the  actual  training  environment 
and  the  experimental  group  will  be  pre-exposed  to  the  training  environment  using  a  PC- 
based  virtual  representation  of  the  environment.  This  research  will  serve  as  benchmark 
for  future  guidance  on  developing  and  implementing  PC-based  virtual  environments  into 
game -based  training  systems  to  meet  military  training  requirements. 
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II.  BACKGROUND 


The  ability  for  an  individual  to  acclimate  himself  to  the  interior  physical  layout  of 
a  real  world  environment  is  essential  in  order  to  successfully  complete  a  task  or  mission 
in  the  military.  From  shipboard  Navy  firefighters,  to  dismounted  infantry  Soldiers, 
acquiring  the  necessary  spatial  orientation  and  familiarity  of  the  physical  layout  of  a  new 
or  unfamiliar  environment  can  mean  the  difference  between  life  and  death.  The  most 
efficient  way  to  leam  the  physical  layout  and  spatially  acclimate  someone  to  a  new  or 
unfamiliar  environment  is  by  actually  immersing  that  person  in  that  real  world 
environment.  However,  with  many  military  applications,  that  maybe  impractical, 
expensive,  or  in  some  case  dangerous.  When  the  location  is  unavailable,  other  methods 
have  been  used  to  replace  the  real  location,  such  as  mock-ups,  maps,  models  or  even 
pictures  of  the  actual  environment.  Again,  this  approach  may  prove  to  be  impractical 
and/or  expensive  to  conduct  in  some  cases.  Additionally,  this  requires  effort  that  could  be 
better  focused  on  other  aspects  of  mission  accomplishment. 

With  the  advent  of  technological  improvements  over  the  last  decade  in  computer 
graphics,  processor  speed,  and  enriched  display  devices;  virtual  environment  (VE) 
technology  has  proliferated  to  a  point  of  greatly  enhancing  training  for  individuals  as  well 
as  teams.  For  example,  the  use  of  virtual  environments  in  shipboard  firefighting  has 
become  a  top  consideration  in  US  Navy  with  respect  to  how  best  to  train  firefighters. 
While  training  firefighting  skills  have  been  the  main  focus  for  using  virtual 
environments,  the  ability  for  a  firefighter  to  acquire  a  requisite  level  of  spatial  orientation 
and  familiarity  of  an  impending  real  world  environment  is  as  important.  He  will  be 
expected  to  successfully  navigate  through  the  many  spaces  of  the  environment  in  order 
too  extinguish  a  fire  or  rescue  a  fallen  crew  member.  This  expectation  is  a  paramount 
consideration  when  developing  virtual  environments  trainers  and  simulators. 

Although  there  are  many  variables  to  consider  when  developing  a  VE  in  order  for 
it  to  be  an  effective  training  tool,  the  concept  of  spatial  knowledge  acquisition  and  its 
effects  on  navigation  is  perhaps  of  the  most  important.  If  a  VE  environment  does  not 
afford  the  spatial  knowledge  or  environment  familiarity  for  a  user  to  successfully 
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navigate  through  the  real  world  environment,  then  training  of  the  kind  mentioned  above 
can  not  take  place.  According  to  Darken  et  al,  it  has  often  been  suggested  that  due  to  its 
inherent  spatial  nature,  a  VE  might  be  a  powerful  tool  for  spatial  knowledge  acquisition 
of  the  real  environment,  as  opposed  to  the  use  of  maps  or  some  other  two-dimensional, 
symbolic  medium  (1998). 

One  of  the  first  studies  to  show  that  a  virtual  environment  could  be  useful  for 
spatial  knowledge  acquisition,  Witmer  et  al  (1995)  compared  a  virtual  environment  to  the 
real  world  in  an  architectural  walkthrough  application.  Their  participants  were  not  experts 
on  the  task  domain  and  used  an  immersive  head-mounted  display  system  which 
employed  gazed-directed  movement  control. 

The  experiment  was  divided  into  four  phases:  Phase  one,  individual  assessment, 
involved  self-reporting  questionnaires  which  centered  on  participant’s  perceived  sense  of 
direction  and  navigation  experience.  In  phase  two,  study,  each  participant  was  given 
fifteen  minutes  to  study  written  step-by-step  route  directions  and  color  photographs  of 
landmarks.  Some  of  the  participants  were  also  given  a  third  study  aid,  which  consisted  of 
a  map  of  the  real  environment.  In  phase  three,  rehearsal,  each  of  the  participants  was 
given  an  opportunity  to  rehearse  their  route  in  either  the  real  or  virtual  environment, 
depending  on  the  group  they  were  assigned  to.  Each  participant  had  to  identify  six 
landmarks  on  the  route.  Any  wrong  turns  and  misidentified  landmarks  made  by  a 
participant  were  immediately  corrected  for  by  the  researchers.  In  the  final  phase,  testing, 
participants  replicated  the  route  they  had  rehearsed  and  asked  to  identify  the  six 
landmarks  they  had  studied  in  the  real  building  environment.  Replication  of  the  route  in 
the  real  building  was  measured  as  attempted  wrong  turns,  time  to  traverse  route,  distance 
to  traverse  route,  and  number  of  misidentified  landmarks.  Participants  were  also  asked  to 
draw  a  line  on  a  map  from  their  known  location  to  an  unseen  target  to  test  their 
knowledge  of  the  configuration. 

This  study  showed  that  route  knowledge  could  be  effectively  trained  to  some 
degree  using  virtual  environment  technology.  Exposure  time  to  the  VE  was  limited 
however,  which  reduced  the  effects  that  it  may  have  had  on  developing  survey 
knowledge. 
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Along  the  same  lines,  Darken  and  Banker  (1998)  used  a  VE  to  study  how 
it  might  be  used  in  addition  to  traditional  real  environment  familiarization  methods.  In 
their  research,  they  looked  at  three  different  groups:  The  first  group  was  restricted  to  only 
using  a  map,  the  second  group  used  a  VE  in  conjunction  with  a  map,  and  the  third  group 
was  restricted  to  using  only  the  real  environment  augmented  with  a  map.  Participants 
interacted  with  the  environment  through  a  keyboard  and  conventional  desktop  display. 
Participants  in  this  study  were  experts  on  the  task  domain  and  the  environment  involved  a 
large  nature  region  of  central  California. 

The  experiment  consisted  of  two  phases.  In  phase  one,  planning  and 
rehearsal,  participants  studied  and  plan  a  route  from  a  starting  point  through  nine 
successive  control  points,  culminating  with  each  participant  having  to  draw  their  planned 
route  on  a  map.  In  the  next  phase,  testing,  participants  executed  their  planned  route 
without  the  assistance  of  the  map  or  a  compass.  The  performance  measures  were  quantity 
of  unplanned  deviations  from  their  planned  route  and  total  distance  traveled. 

This  study  differs  from  Witmer  et  al’s  earlier  study  in  that  the  participants 
planned  and  practiced  their  own  route,  not  a  route  that  was  given.  Additionally,  the 
participants  were  experts  on  the  task  domain  and  the  time  of  exposure  to  the  VE  was  four 
times  longer. 

The  results  of  this  study  showed  that  beginners  to  the  task  domain  were 
not  assisted  by  the  VE  exposure;  intennediate  users  benefited  the  most  from  the  VE 
exposure;  advanced  users  did  not  show  any  improvement.  To  explain  this,  Darken  and 
Banker  point  out  that  beginners  lack  the  level  of  knowledge  on  the  task  domain  to  be 
assisted  by  the  VE;  advanced  users  are  so  experienced  in  the  task  domain  that  the  VE  did 
little  to  improve  their  perfonnance.  They  concluded  that  given  a  short  train-up  time,  maps 
still  seem  to  be  the  best  alternative  to  spatial  knowledge  acquisition. 

In  a  study  conducted  by  Koh  et  al  (2000),  development  of  configurational 
knowledge  using  an  architectural  environment  was  investigated.  In  the  study,  the 
researchers  compared  four  different  groups.  The  first  group  was  the  real  world  group,  the 
second  group  used  an  immersive  VE,  consisting  of  a  head  mounted  display  (HMD),  the 
third  group  used  a  desktop  VE,  consisting  of  a  typical  computer  monitor,  and  the  fourth 
group  used  a  hand  held  World-In-Miniature  device.  Each  of  the  participants  was 
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randomly  assigned  to  each  of  the  four  groups  and  there  were  no  expert  knowledge  on  the 
task  domain.  The  users  of  the  two  VE  groups  controlled  their  motion  speed  and  direction 
through  the  use  of  a  joystick. 

The  actual  experiment  was  executed  in  three  phases.  The  administration 
phase  consisted  of  infonning  the  participants  of  the  task  that  would  be  completed,  but 
specific  target  infonnation  was  not  disclosed.  During  this  first  phase,  participants  using 
the  VEs  were  also  allowed  time  to  become  familiar  with  their  respective  interfaces.  In  the 
next  phase,  training,  participants  were  given  ten  minutes  to  explore  either  the  VE  or  the 
real  environment  depending  on  the  group  they  were  assigned  for  the  experiment.  In  the 
final  testing  phase  of  the  experiment,  participant’s  configurational  knowledge  was  tested 
by  asking  them  for  bearing  and  distance  estimations  to  unseen  targets.  The  results  of  this 
study  showed  that  configurational  knowledge  could  be  just  as  effectively  training  in  a  VE 
as  in  a  real  world  environment. 

In  1998,  Waller  et  al  conducted  a  study  which  used  six  different  type  of 
treatment  condition  to  further  answer  questions  about  VE  ability  to  acquire  spatial 
knowledge.  The  treatment  groups  were  divided  into  a  real  world  group,  a  group  that  had 
no-study,  and  three  VE  groups  (desktop,  short-duration  immersion,  and  long-duration 
immersion).  The  participants  in  the  study  were  non-experts  on  the  task  domain  and  the 
environment  used  in  the  experiment  was  a  maze  with  targets  placed  a  specific  locations. 
Each  of  the  VE  groups  used  a  joystick  for  motion  control  in  their  respective  environment 
and  the  immersive  groups  used  a  HMD.  The  short-duration  immersive  group  was  allowed 
twelve  minutes  to  practice  in  the  VE  and  the  long-duration  group  was  allowed  thirty 
minutes  of  practice  time. 

The  experiment  was  executed  in  four  phases.  In  the  first  phase,  the 
participants  were  administered  a  Guiford-Zimmerman  spatial  abilities  test.  The  second 
phase  consisted  of  allowing  the  participants  to  interweave  practice  and  testing  in  the 
environments  for  a  total  of  six  runs.  In  the  third  phase,  the  researchers  altered  the  maze 
configuration  so  that  portions  of  the  learning  maze  were  blocked  and  the  participants  had 
to  find  a  new  route  to  traverse,  this  tested  their  survey  knowledge.  Perfonnance  measures 
for  the  previous  two  phase  included  time  to  traverse  a  route  and  number  of  times  a 
participant  bumped  into  a  wall.  In  the  final  phase  participants  complete  a  configurational 
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knowledge  test  of  the  maze  environment.  This  study  gave  credence  to  and  showed  that 
given  enough  exposure  time,  training  in  a  VE  could  be  more  effective  in  training  spatial 
knowledge  then  the  real  world  environment. 

Although  all  of  the  previous  research  have  shown  that  training  in  a  VE  can 
be  as  effective  as  training  in  a  real  environment,  these  studies  are  of  a  general  nature  and 
may  not  transfer  to  a  specific  task  domains.  Research  needs  to  be  completed  on  specific 
task  domains  in  order  to  investigate  feasibility  of  training  that  problem  domain  in  a  VE. 

Accordingly,  in  the  task  domain  of  shipboard  firefighting  the  research  is 
limited  with  only  two  such  studies  completed  to  date.  The  first  such  study  conducted  by 
Bliss  et  al  (1995)  showed  that  fire  fighters  exposed  to  a  virtual  environment  of  an  office 
building  improved  their  navigational  abilities  through  the  building  when  conducting 
search  and  rescue  operations. 

The  other  such  study  specifically  tied  to  training  spatial  knowledge  in  firefighters 
was  conducted  by  Tate,  Sibert,  and  King  (1997).  In  their  study  the  researchers  explored 
the  feasibility  of  using  immersive  VE  technology  as  a  tool  for  shipboard  firefighting 
training  and  mission  rehearsal.  The  researchers  modeled  portions  of  the  Navy’s  ex-USS 
SHAD  WELL  (LSD- 15),  a  former  amphibious  ship  now  used  as  a  firefighting  lab  in  a 
VE.  The  system  employed  an  immersive,  head-mounted  design  (HMD)  device  to  view 
the  virtual  3D  model  of  the  environment  and  a  3D  joystick  for  motion  control  and 
interacting  with  the  virtual  environment.  The  participants  in  the  study  were  divided  into 
two  groups,  a  traditional  and  a  VE  group;  the  participants  were  experts  on  the  task  and 
none  were  familiar  with  the  real  environment. 

The  study  was  divided  into  two  phases.  Phase  one  consisted  of  a  navigational  task 
that  did  not  involve  firefighting  skills.  The  task  was  to  traverse  a  specified  path  through  a 
simulated  smoke  filled  ship  environment.  Perfonnance  measures  included  time  to 
traverse  the  route  and  number  of  wrong  turns  taken.  In  the  second  phase,  participants 
were  required  to  retrieve  specific  gear,  perfonn  standard  firefighting  procedures,  and  lead 
the  firefighting  team  to  extinguish  an  actual  shipboard  fire.  The  purpose  of  this  phase  was 
designed  to  evaluate  whether  or  not  a  VE  helps  fire  fighters  actually  extinguish  a  fire 
faster  than  firefighters  without  VE  training. 
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The  results  from  this  study  showed  that  there  was  a  measurable  improvement  in 
the  performance  of  firefighters  that  used  VE  for  mission  rehearsal  over  firefighters 
without  VE  in  both  phases  of  the  test.  Additionally,  the  study  showed  that  VE’s  can  be 
effectively  used  for  training  shipboard  familiarity  as  well.  Further,  Tate  et  al  sighted  that 
VE’s  provide  a  flexible  environment  where  a  firefighter  can  not  only  learn  an  unfamiliar 
part  of  the  ship,  but  also  practice  tactics  and  procedures  for  fighting  a  live  fire. 

This  and  other  previous  studies  in  the  area  of  training  using  virtual  environments 
have  proved  to  be  a  monumental  step  in  showing  how  such  technology  could  be 
successfully  employed  to  train  shipboard  firefighting  and  other  general  task 
accomplishments,  more  importantly  however,  spatial  knowledge  acquisition  of  unfamiliar 
environments.  However,  these  studies  were  based  on  training  employing  the  concept  of 
immersive  type  of  technology,  such  as  HMDs  or  CAVEs.  Toward  this  end,  additional 
research  needs  to  be  conducted  on  training  firefighters  and  other  task  domains  leveraging 
VE  technologies  designed  around  typical  display  devices,  allowing  interaction  between 
individual  and  the  VE  through  traditional  methods  (i.e.  standard  keyboard  &  mouse 
devices).  Measuring  the  effects  that  this  might  have  on  spatial  knowledge  acquisition  is 
of  great  importance  as  well. 
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III.  METHODOLOGY 


A.  EXPERIMENT  OVERVIEW 

The  goal  of  this  experiment  was  to  determine  if  using  a  PC-based  virtual 
replication  or  model  of  a  real  world  small-scale  environment  had  a  positive  affect  on  an 
individual’s  ability  to  acquire  a  sufficient  level  of  spatial  orientation  and  familiarity  with 
the  physical  layout  of  the  real  world  environment  in  order  to  successfully  perform  a  set  of 
performance  measures  in  the  real  environment.  The  following  section  describes  in  more 
detail  the  phases  and  procedures  used  to  conduct  the  experiment. 

The  experiment  was  conducted  in  San  Diego,  CA  at  the  US  Navy’s  Damage 
Control  and  Fire  Fighting  School.  The  venue  was  the  F-l  Fire  Fighting  Training  (see 
Figure  1).  Participants  were  Sailors  currently  assigned  to  the  Navy  Pre-Commissioning 
Detachment  (PCD)  and  were  randomly  selected  to  participate  in  the  experiment.  All 
participants  were  briefed  on  the  experiment  and  completed  the  required  consent  form. 
(See  Appendix  A  and  B).  Participants  were  also  given  a  brief  survey  to  gather  key 
demographic  information  and  general  work  experience.  (See  Appendix  E). 


Figure  1.  F-l  Fire  Fighting  Trainer,  San  Diego,  CA 
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Participants  were  randomly  divided  into  two  separate  groups  for  the  entire 
experiment:  a  real  environment  group  and  a  virtual  environment  group.  Each  group 
explored  the  spaces  within  their  respective  environments.  Upon  completion  of  the 
environment  exploration,  each  participant  reconstructed  from  memory  a  detailed  map  of 
the  environment  in  which  they  explored.  Each  participant  then  performed  a  set  of 
retrieval  tasks  within  the  real  environment.  Participants  were  asked  to  retrieve  a  certain 
colored  flag  in  a  particular  space  located  on  a  specific  piece  of  equipment  within  the  real 
world  environment. 

The  first  test  was  scored  based  on  the  correct  number  of  rooms  sketched  and  the 
correct  location  of  equipment  inside  each  of  the  rooms.  The  retrieval  tests  were  scored 
using  the  time  to  successfully  retrieve  the  correct  colored  flag. 

Upon  completion  of  the  experiment,  each  participant  was  debriefed  and  thanked 
for  their  participation  in  the  experiment. 

B.  EXPERIMENTAL  DESIGN 

The  experiment  conducted  was  a  ‘between  subjects’  design.  The  virtual 
environment  group  was  pre-exposed  to  the  virtual  replication  of  the  real  world 
environment  before  conducting  their  tests  in  the  real  environment.  The  real  environment 
group  was  pre-exposed  to  the  real  environment  before  conducting  their  test  in  that  same 
environment. 

1.  Participants 

There  participants  for  the  experiment  consisted  of  14  US  Navy  active  duty 
enlisted  sailors.  They  ranged  in  pay  grade  from  E2  to  E6.  The  age  range  was  18  to  29 
years  with  an  average  age  of  23.  The  participants  all  had  varying  degrees  of  experience  in 
a  shipboard  environment  with  an  average  time  in  service  at  3.5  years.  Gender  was  not  a 
consideration  in  the  study  due  to  the  fact  that  only  one  of  the  fourteen  participants  was 
female.  Of  the  participants,  43%  of  them  were  in  non-technical  ratings  or  jobs.  None  of 
the  subjects  had  prior  knowledge  as  too  the  physical  layout  of  the  F-l  Fire  Fighting 
Trainer. 
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2.  Environment  Exposure 

Each  group  was  systematically  exposed  to  their  respective  environments  before 
any  testing  occurred  in  the  real  world  environment.  The  virtual  environment  group  was 
given  a  short  training  session  on  how  to  maneuver  within  the  virtual  model  located  on  the 
PC.  After  completion  of  the  training  session,  participants  explored  each  room  within  the 
model.  Time  within  each  space  was  limited  to  five  minutes.  Within  each  room 
participants  we  given  a  guided  tour  (see  Appendix  C).  Their  attention  was  drawn  to 
specific  aspects  within  each  space,  including  the  location  of  specific  pieces  of  equipment. 
The  real  environment  group  was  given  exactly  the  same  guided  tour  during  their 
exploration  of  the  real  world  environment.  The  time  spend  in  each  space  was  consistent 
with  the  time  for  the  virtual  environment  group. 

3.  Order  of  Test 

Upon  completion  of  the  exploration  phase  of  the  experiment,  each  participant 
completed  a  detailed  sketch  of  the  environment  in  which  they  toured.  This  test  was 
completed  prior  to  conducting  the  retrieval  tasks  to  avoid  any  learning  effects  after 
exposure  to  the  real  environment  during  the  second  phase  of  testing.  The  measure  used  to 
quantify  the  performance  of  each  group  was  a  combined  score  based  on  the  number  of 
rooms  draw  and  the  correct  placement  of  equipment  within  a  particular  space.  Sketches 
were  drawn  on  a  ‘spatial  exam’  form  designed  specifically  for  the  experiment,  which  is 
provided  in  Appendix  B.  It  is  important  to  note  that  artistry  and  scale  were  not  considered 
when  scoring  the  exam.  Only  placement  of  rooms  and  equipment  was  considered  in  the 
scoring  process.  Figure  2  shows  a  ‘ground  truth’  depiction  of  the  real  world  environment 
and  was  used  to  score  the  participant’s  sketches. 
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The  second  tests  consist  of  randomly  assigning  each  participant  two  separate,  but 
equivalent,  retrieval  tasks.  In  each  task  participants  had  to  retrieve  a  specific  colored  flag 
located  within  one  of  the  spaces  in  the  real  world  environment.  Figure  2  also  shows  the 
location  and  color  of  each  of  the  flags  and  their  location  used  for  the  second  test.  The 
measure  used  to  quantify  the  performance  of  each  participant  was  the  time  to  retrieve  the 
assigned  flag. 

C.  VENUE 

1.  Real  World  Environment 

The  environment  selected  to  conduct  the  experiment  was  the  US  Navy’s  F-l  Fire 
Fighting  Trainer,  located  in  San  Diego,  CA.  (see  Figure  1).  The  selection  of  this  venue 
was  two-fold.  Firstly,  it  represented  a  typical  shipboard  environment  in  which  US  Navy 
personnel  would  be  expected  to  be  familiar  with  in  order  to  effectively  conduct  training. 
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Secondly,  once  modeled  in  a  virtual  environment  for  this  experiment,  it  could  be 
expanded  for  future  studies  of  this  kind. 

2.  Virtual  Environment 

The  virtual  environment  was  an  exact  3D  graphical  representation  of  the  F-l  Fire 
Fighting  Trainer.  The  model  was  created  using  3D  Studio  Max  by  Matt  Prichard,  a 
modeler  and  artist  at  the  Naval  Postgraduate  School  (NPS).  The  motion  model  was 
designed  by  Eric  Johnson  and  Perry  McDowell  also  at  NPS.  The  virtual  model  was 
validated  by  the  Officer  in  Charge  and  the  staff  instructors  at  the  Damage  Control  and 
Fire  Fighting  School  in  San  Diego,  CA.  Figure  3  shows  the  first  level  of  the  virtual  F-l 
Fire  Fighting  Trainer  model. 


Figure  3.  First  Level  of  Virtual  F-l  Trainer  Model 


Figure  4  below  compares  the  deep-fat  fryer  room  in  the  real  environment  (left) 
and  in  the  virtual  environment  (right). 

The  model  was  run  on  a  standard  desktop  computer  with  a  Pentium  4  processor 
and  512  Mbytes  of  RAM.  The  interface  was  a  standard  keyboard  and  mouse.  Participants 
interacted  with  the  virtual  model  through  mouse  and  keyboard.  The  mouse  was  used  for 
lateral  motion  and  for  looking  up  and  down.  Keyboard  keys  ‘w’,  ‘s’,  ‘a’,  and  ‘d’,  were 
used  to  step  forward,  backward,  left,  or  right,  respectively,  the  standard  PC  first  person 
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shooter  motion  model.  Motion  through  the  virtual  environment  was  held  at  a  speed 
equivalent  to  the  walking  pace  of  the  real  environment  group. 

The  display  used  in  the  experiment  was  a  standard  17  inch  LCD  flat-panel.  The 
configuration  of  hardware  was  chosen  based  on  standard  equipment  readily  available  to 
most  Navy  personnel  in  the  fleet.  The  idea  was  not  to  introduce  any  new  hardware 
configurations  that  were  not  in  common  use. 


Figure  4.  Real  vs.  Virtual  Environment 


D.  MEASUREMENT  SELECTION 

The  performance  measures  selected  for  the  experiment  were  based  on  a  detailed 
review  of  the  literature  with  respect  to  completion  of  task  in  virtual  and  real 
environments.  Based  on  the  literature  reviewed  there  are  three  main  components  that 
makeup  spatial  knowledge  and  would  be  measured  in  the  experiment:  Landmark;  Route; 
and  Survey  (Karahan  2000). 
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Although  the  efficacy  of  each  component,  according  to  Karahan,  can  be  evaluated 
using  several  different  performance  measures  the  following  were  selected  for  the 
experiment. 

1.  Sketch  Mapping  and  Map  Placement 

In  order  to  measure  a  participant’s  familiarity,  or  survey  knowledge,  of  the  real 
environment  after  being  exposed  first  to  a  virtual  replication  of  that  environment,  a 
combination  of  sketch  mapping  and  map  placement  were  used  in  the  experiment.  These 
performance  measures  were  selected  in  order  to  quantify  the  mental  or  cognitive  map  that 
participants  developed  of  the  real  environment  after  first  being  exposed  to  either  a  virtual 
replication  of  the  environment  or  the  actual  environment  itself.  Although  there  are  some 
who  feel  that  the  use  of  such  a  performance  measure  is  problematic  since  they  are 
difficult  to  score  and  may  under  estimate  the  actual  knowledge  acquired  (Siegel  1981), 
the  experiment  revealed  that  by  combining  the  sketch  mapping  and  map  placement  tasks 
and  scoping  it  within  the  context  of  an  environment  without  a  high  level  of  detail,  it  is 
quite  useful  in  gauging  a  participant’s  familiarity  or  survey  knowledge  of  the 
environment’s  physical  layout. 

2.  Retrieval  Task 

To  measure  a  participant’s  landmark  and  route  knowledge,  the  choice  was  made 
to  combine  both  measures  into  evaluating  the  time  to  retrieve  a  certain  colored  flag  in  a 
specific  location  within  the  real  environment.  Although  Karahan’ s  work  on  performance 
measures  in  virtual  and  real  world  environments  briefly  addresses  a  walking  task  as  not  a 
viable  measure  to  quantify  perfonnance,  the  retrieval  task  used  in  the  experiment  proved 
to  be  a  viable  measure  to  quantify  a  participant’s  performance  in  a  real  or  virtual 
environment. 

Based  upon  previous  research  and  in  this  experiment,  there  is  evidence  that  the 
retrieval  task  can  incorporate  the  many  of  the  measures  Karahan  points  to  in  order  to 
measure  landmark  and  route  knowledge.  In  order  to  retrieve  an  object  from  a  specific 
location  within  an  environment,  the  participant  must  be  able  to  apply  the  necessary 
landmark  and  route  knowledge  of  the  environment  to  successfully  navigate  though  the 
environment,  obtain  the  object  and  return  with  it.  If  that  is  not  the  case,  the  participant  is 
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considered  lost  and  his  time  to  retrieve  the  object  will  reflect  that.  In  this  experiment,  one 
participant  became  lost  or  disoriented  within  the  environment  and  the  time  to  retrieve  his 
object  was  much  greater  than  the  other  participants  in  his  group. 
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IV.  ANALYSIS 


A.  RESULTS 

1.  General  Information 

The  experiment  is  designed  to  test  a  hypothesis  concerning  the  correction  between 
spatial  orientation  and  familiarity  with  the  layout  of  a  real  world  environment  after  pre¬ 
exposure  to  a  PC-based  virtual  replication  or  model  of  that  environment.  To  determine 
quantify  participant  perfonnance,  their  times  to  search  for  and  retrieve  two  separately 
located  flags  in  the  real  environment  and  a  combined  score  assessing  their  ability  to 
reconstruct  the  layout  of  the  real  environment  using  a  sketch  were  used. 

a.  Hypothesis 

Participants  exposed  to  a  3D  virtual  replication  of  a  real  world 
environment  will  have  the  same  spatial  orientation  and  familiarity  as  participants  pre¬ 
exposed  only  to  the  real  world  environment. 

1.  A  participant  who  has  gained  a  level  of  familiarity  with  the  real 
environment  will  be  able  to  reconstruct  the  physical  layout  of  that  environment  in  the 
form  of  a  sketch. 

2.  A  participant  who  has  gained  a  level  of  spatial  orientation  will  be  able 
to  search  for  and  retrieve  a  specified  flag  located  in  one  of  the  real  environment  spaces  in 
a  time  equivalent  between  groups. 

2.  Significance  Level 

The  experiment  involved  a  relatively  small  sample  size;  however,  to  keep  consist 
with  previous  studies  involving  human  participants  and  virtual  environments  an  a  value 
of  .05  was  used  for  all  analysis  to  detennine  statistical  significance  between  the  virtual 
and  real  environment  groups. 

There  were  five  different  search  and  retrieve  tasks  in  the  second  testing  phase  of 
the  experiment.  Participants  were  randomly  assigned  to  complete  two  of  the  five  tasks. 
The  assumption  was  made  that  there  was  no  difference  between  the  five  task  based  on 
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results  of  an  ANOVA  using  the  independent  task  to  determine  main  effects.  This 
assumption  is  an  important  first  step  in  the  overall  analysis  in  order  look  at  only  the 
significance  between  groups. 

The  data  in  the  succeeding  section  is  presented  using  box  plots,  scatter  plots,  and 
frequency  plots. 

3.  Spatial  Exam  Results 

To  compare  participant’s  acquisition  of  survey  knowledge  between  groups,  each 
participant  was  instructed  to  sketch  a  map  of  the  environment  in  which  they  were 
exposed  to  during  the  guided  tour  phase  of  the  experiment.  In  addition,  participants  were 
also  instructed  to  place  specific  items  that  they  viewed  in  the  environment  in  the  correct 
location  on  their  map. 

The  box  plot  in  Figure  5  displays  the  spatial  exam  scores  for  each  of  the  two 
groups.  The  mean  score  for  the  real  group  was  greater  (M=20.29,  s=1.604)  than  the  mean 
score  for  the  virtual  group  (M=10.33,  s=3.077).  A  summary  of  these  results  is  presented 
in  Table  1.  An  independent  sample  test  indicated  that  there  was  a  statistical  significance 
between  the  groups’  ability  to  recall  and  reconstruct  the  spatial  layout  of  the  real 
environment,  t(l  1)=7.489,  p=.00. 
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Spatial  Exam  Scores 
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Figure  5.  Spatial  Exam  Scores  for  Each  Group 
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1  Real  2  Virtual 

Group 


Group 

N 

Mean 

Std.  Deviation 

1  -  Real 

7 

20.29 

1.604 

2  -  Virtual 

7 

10.33 

3.077 

Table  1. 


Summary  Statistics  for  Spatial  Exam  Scores 


4.  Retrieval  Tasks  Results 

To  compare  acquired  landmark  and  route  knowledge  between  the  real  and  virtual 
environment  groups  a  set  of  retrieval  task  were  performed  in  the  real  environment.  The 
bar  chart  in  Figure  6  graphically  displays  each  participant’s  time  to  accomplish  both  task 
one  and  task  two.  The  real  environment  group  on  the  graph  are  numbered  one  to  seven, 
with  participants  in  group  two,  the  virtual  environment  group,  numbered  eight  through 
fourteen.  From  the  graph,  a  general  observation  of  the  data  indicates  that  overall, 
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participants  in  the  real  environment  group  appear  to  have  perfonned  better,  having  a 
shorter  time  on  task  one  then  group  two.  On  the  other  hand,  both  groups  appear  to  have 
performed  at  or  near  the  same  level  during  performance  of  task  2. 

During  the  initial  stages  of  data  analysis,  participant  eleven  in  group  two  was 
determined  to  be  an  excessive  outlier.  Closer  examination  of  participant  eleven  revealed 
that  he  was  not  indicative  of  the  target  population  for  this  study;  therefore  participant 
eleven’s  data  was  extracted  from  the  remaining  data  analysis. 


Performance  Times  for  Task  1  and  2 


Participant  Number 

Figure  6.  Task  One  and  Two  Retrieval  Times  for  Each  Participant 


The  group  times  to  successfully  retrieve  the  first  flag  are  displayed  in  a  box  plot 
in  Figure  7.  The  mean  time  for  the  real  group  was  less  (M=24.71,  s=5.992)  than  mean 
time  for  the  virtual  group  (M=31.83,  s=13.848).  A  summary  of  these  results  is  presented 
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in  Table  2  below.  An  independent  sample  test  indicated  that  there  was  not  a  statistically 
significance  difference  between  the  group’s  ability  to  fonnulate  the  required  spatial 
orientation  or  knowledge  of  the  real  environment  in  order  to  successfully  retrieve  their 
first  flag,  t(l  1)=-1 .238,  p=. 241. 


Performance  Times  for  Task  1 


1  Real  2  Virtual 

Group 


Figure  7.  Time  to  Retrieve  First  Flag 


Group 

N 

Mean 

Std.  Deviation 

1  -  Real 

7 

24.71 

5.992 

2  -  Virtual 

6 

31.83 

13.848 

Table  2.  Summary  Statistics  for  Task  1 


The  group  times  to  successfully  retrieve  the  second  flag  are  displayed  in  a  box 
plot  in  Figure  8.  The  mean  time  for  the  real  group  was  less  (M=14.29,  s=5.992)  than 
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mean  time  for  the  virtual  group  (M=18.50,  s=5.468).  A  summary  of  these  results  is 
presented  in  Table  3  below.  An  independent  sample  test  indicated  that  there  was  not  a 
statistical  significance  between  the  group’s  ability  to  formulate  the  required  spatial 
orientation  or  knowledge  of  the  real  environment  in  order  to  successfully  retrieve  their 
second  flag,  t(  1 1)=- 1 .683,  p=.  121. 

These  results  indicate  that  after  each  group  was  exposed  to  the  real  environment 
during  task  one,  participants  in  each  group  performed  better  overall.  However,  one  group 
did  not  do  significantly  better  than  the  other  group. 
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Figure  8.  Time  to  Retrieve  Second  Flag 


Group 

N 

Mean 

Std.  Deviation 

1  -  Real 

7 

14.29 

3.498 

2  -  Virtual 

6 

18.50 

5.468 

Table  3.  Summary  Statistics  for  Task  2 
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Looking  at  the  relationship  displayed  in  a  scatter  plot  graph,  Figure  9,  between 
exam  scores  and  retrieval  times  for  group  1,  there  was  not  a  relationship  (r=-.099,  p>.05). 
However,  looking  at  the  relationship  between  group  exam  scores  and  retrieval  tests  for 
group  two,  there  was  a  strong  negative  correlation  between  exam  scores  and  retrieval 
times  (r=-.829,  p=.042).  These  results  indicate  that  participants  in  group  two  who 
performed  lower  on  the  spatial  exam  had  a  more  difficult  time  finding  their  flags  in  the 
real  environment. 


Performance  Times  vs.  Exam  Scores 
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Figure  9.  Scatter  Plot  Graph  for  Retrieval  Times  and  Spatial  Exam  Scores 

B.  DISCUSSION 

1.  General  Discussion 

The  general  results  of  the  experiment  indicated  that  there  was  not  a  significant 
difference  between  group  ability  to  acquire  the  requisite  level  of  spatial  orientation  in 
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order  to  successfully  retrieve  the  correct  object  from  the  real  world  environment;  whether 
they  were  pre-exposed  to  the  PC-based  virtual  environment  or  the  real  world 
environment.  These  results  are  congruent  with  much  of  the  literature  on  using  virtual 
environment  technology  to  train  real  world  spatial  orientation  with  respect  to  landmark 
and  route  knowledge.  The  results  for  testing  survey  knowledge  acquisition,  however,  did 
not  match  previous  research  results  in  this  area.  There  are  several  contributing  factors  that 
may  have  attributed  to  the  above  result  from  the  experiment. 

The  succeeding  section  discusses  environment,  participant,  and  transfer  of 
training  issues  that  are  possible  contributing  factors  which,  if  taken  into  account  for 
future  work,  may  produce  stronger,  more  conclusive  results. 

a.  Environment  Issues 

In  order  to  train  an  individual  to  successfully  navigate  through  a  real 
environment  they  must  posses  an  adequate  level  of  landmark  and  route  knowledge.  This 
study  showed  that  it  was  possible  to  transfer  that  knowledge  to  an  individual  using  a  PC- 
based  virtual  representation  of  the  environment.  However,  the  data  also  indicated  that 
participants  in  the  virtual  group  did  not  acquire  spatial  knowledge  at  the  same  level  as 
those  participants  pre-exposed  to  the  real  world  environment.  This  maybe  due  in  part  to  a 
fidelity  issue  associated  with  the  virtual  model.  Although  virtual  environment  technology 
has  vastly  improved  over  the  past  several  years,  currently  it  is  impossible  to  exactly 
replicate  an  environment  to  the  point  that  there  is  absolutely  no  difference  between  the 
virtual  and  real  world  environments. 

Fidelity  issues  in  the  virtual  environment  may  have  also  played  a  role  in 
the  test  group’s  ability  to  acquire  the  same  level  of  survey  knowledge  that  the  control 
group  acquired.  The  data  in  the  experiment  indicated  that  there  was  a  significant 
difference  between  the  two  groups,  with  the  real  environment  group  doing  on  average 
fifty  percent  better  than  the  virtual  group. 

b.  Participants  Issues 

The  experiment  was  conduct  with  a  relatively  small  sample  size  of 
fourteen.  Additionally,  the  extraction  of  the  outlier  in  the  virtual  group  for  data  analysis 
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reduced  this  number  even  more.  A  study  of  this  type  should  include  on  the  order  of  ten 
times  as  many  participants  in  order  to  make  stronger  correlations  between  using  a  PC- 
based  virtual  environment  to  train  spatial  knowledge  and  environment  familiarity. 

As  in  previous  studies  as  well  as  the  experiment  present  here,  population 
samples  are  restricted  to  a  specific  domain.  This  experiment  randomly  selected 
participants  from  a  population  of  U.S.  Navy  enlisted  personnel.  Previous  studies  have 
used  foreign  and  U.S.  officers.  Each  of  these  population  samples  has  varying  degrees  of 
experience  relating  to  virtual  or  computed  based  training.  In  order  to  make  stronger 
generalization  of  the  efficacy  of  using  PC-based  or  any  other  types  of  virtual  training 
environments  in  the  military,  a  broader  sample  of  the  military  population  at  larger  should 
be  used. 

c.  Transfer  of  Training  Issues 

Positive  training  transfer  is  another  important  aspect  that  plays  a  critical 
role  in  the  efficacy  of  this  type  of  study.  The  participants’  ability  to  properly  transfer  the 
spatial  orientation  they  learned  from  exposure  to  the  virtual  environment  may  have  been 
impeded  by  the  shear  novelty  of  such  an  environment.  The  virtual  environment,  for  most 
of  the  participants,  was  a  new  device  in  which  to  view  some  aspect  of  the  real  world.  The 
real  world  participants  do  not  readily  experience  this  novelty  factor.  In  addition, 
participants  using  the  virtual  replication  of  the  F-l  trainer  did  not  experience  a  sense  of 
immersion  within  the  environment  as  those  pre-exposed  to  the  actual  training 
environment  had  experienced. 
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V.  CONCLUSION  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

1.  General 

The  focus  of  this  thesis  was  to  conduct  an  experiment  to  detennine  the  efficacy  of 
using  a  PC-based  virtual  environment  to  aid  participants  in  acquiring  a  requisite  level  of 
spatial  orientation  and  environment  familiarity  in  order  to  perform  task  at  the  same  level 
participants  pre-exposed  to  the  real  world  environment.  Previous  studies  in  this  area  have 
been  conducted  using  other  virtual  environment  technologies,  such  as  CAVES  or  HMDs, 
to  evaluate  perfonnance  in  a  real  world  enviromnent.  In  addition,  those  studies  have  each 
used  different  approaches  to  measuring  that  perfonnance.  The  approach  taken  in  this 
study  was  to  use  a  standard  PC-based  virtual  environment  configuration  and  evaluate 
performance  in  a  real  world  environment  using  measure  that  translate  well  to  real  world 
task  accomplishment. 

The  results  from  the  experiment  were  analyzed  in  order  to  detennine  if  there  was 
a  conelation  between  using  the  PC-based  virtual  environment  and  performing  in  the  real 
world  environment.  The  following  conclusions  stand  to  illustrate  the  efficacy  of  such  an 
environment. 

a.  Participants  pre-exposed  to  the  PC-based  virtual  environment  acquired 
the  same  spatial  orientation  and  familiarity,  with  respect  to  their 
landmark  and  route  knowledge,  as  those  participants  pre-exposed  to 
real  world  environment,  however  not  at  the  same  level. 

b.  Participants  pre-exposed  to  the  PC-based  virtual  environment  did  not 
acquire  the  same  spatial  orientation  and  familiarity,  with  respect  to 
their  survey  knowledge,  as  those  participants  pre-exposed  to  the  real 
world  environment. 

c.  Participants  pre-exposed  to  the  PC-based  virtual  environment 
performed  at  or  near  the  same  level  after  subsequent  exposure  to  the 
real  world  environment. 
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B.  RECOMMENDATIONS 

1.  Different  Media 

While  this  study  only  looked  at  using  a  PC-based  virtual  environment 
configuration,  the  use  of  other  media  delivery  methods,  e.g.,  video  or  still  picture  images, 
should  be  explored.  The  use  of  other  delivery  methods  may  reveal  that  training  in  spatial 
environments,  critical  to  the  transfer  of  training  to  computer  based  methods,  can  be  done 
in  methods  other  than  virtual  environments 

2.  Sample  Size 

This  study  suggest  that  there  is  evidence  that  using  a  PC-based  virtual 
environment  is  as  good  as  the  real  world  environment  when  training  individuals  in 
acquiring  spatial  orientation  and  familiarity  in  a  new  environment.  However,  in  order  to 
state  that  stronger  and  more  conclusively,  future  studies  of  this  type  should  be  conducted 
using  a  more  congruent  and  larger  sample  of  the  general  military  population. 
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APPENDIX  A.  EXPERIMENT  OUTLINE 


1)  In  Brief  /Consent  Form/Pre- Survey 

a)  Time  -  5  Min. 

b)  Location-Firefighting  Training  School  San  Diego,  CA. 

c)  Monitor  -  LT.  Matthew  Molmer 

d)  Materials  -  IRB  Consent,  Privacy  Act,  &  Minimal  Risk  Fonn,  Pencil, 
Introduction  Briefing  Scripts,  Pre-Survey 

2)  PC  Interface  Familiarization  (Virtual  Environment  Group  Only) 

a)  Time  -  5  Min. 

b)  Location-  Firefighting  Training  School  San  Diego,  CA. 

c)  Monitor  -  LT  Matthew  Molmer 

d)  Materials  -  Alienware  Desktop  Computer  System,  Mouse,  Keyboard, 
Virtual  Environment  Group  Briefing  Script 

3)  Guide  Tour  Using  PC-based  Virtual  Environment 

a)  Time  -  3  -  5  minutes  each  Participant 

b)  Location-  Classroom  Firefighting  Training  School  San  Diego,  CA. 

c)  Monitor  -  LT  Matthew  Molmer 

d)  Materials  -  Alienware  Desktop  Computer,  Mouse,  Keyboard,  Virtual 
Environment  Group  Briefing  Script 

e)  Tour  through  Virtual  Spaces: 

(1)  Space  1:  Deep  Fat  Fryer  Space  -  Includes  Power  Panels, 
Gaylord  Hood  &  Activation  Bottle,  Storage  Space 
Entrance. 

(2)  Space  2:  Boiler  Space:  Includes  Auxiliary  Boiler,  Fuel 
Oil  Trip  Value,  Power  Panels,  Battle  Lantern,  Halon 
Bottles  and  Actuator. 
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(3)  Space  3:  Shaft  Alley:  Includes  Shaft,  AFFF  Hose  Reel, 
Power  Panel. 

(4)  Space  4:  Laundry  Space:  Includes  Dryer,  Table,  Power 
Panel. 

4)  Guided  Tour  in  Real  World 

a)  Time  -  5  minutes  per  space  for  each  participant 

b)  Location-  F-l  Firefighting  Trainer  School  San  Diego,  CA. 

c)  Monitor  -  LT  Matthew  Molmer 

d)  Materials  -  F-l  Firefighting  Trainer,  Real  World  Group  Briefing  Script 

e)  Tour  through  Real  Environment: 

(1)  Space  1:  Deep  Fat  Fryer  Space  -  Includes  Power  Panels, 
Gaylord  Hood  &  Activation  Bottle,  Storage  Space 
Entrance. 

(2)  Space  2:  Boiler  Space:  Includes  Auxiliary  Boiler,  Fuel 
Oil  Trip  Value,  Power  Panels,  Battle  Lantern,  Halon 
Bottles  and  Actuator. 

(3)  Space  3:  Shaft  Alley:  Includes  Shaft,  AFFF  Hose  Reel, 
Power  Panel. 

(4)  Space  4:  Laundry  Space:  Includes  Dryer,  Table,  Power 
Panel. 

5)  Administer  spatial  exam 

a)  Time  -  15  minutes  for  each  participant 

b)  Location-  Classroom  Firefighting  Trainer  School  San  Diego,  CA. 

c)  Monitor  -  LT  Matthew  Molmer 

d)  Materials  -  Spatial  exam,  pencils,  stop  watch 

6)  Evaluation  and  Test  Using  Real  Environment 

a)  Time  -  10  -  15  minutes  each  Participant 

b)  Location-  F-l  Firefighting  Trainer  School  San  Diego,  CA. 

c)  Monitor  -  LT  Matthew  Molmer 
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d)  Materials  —  F-l  Firefighting  Trainer,  Testing  and  Evaluation  Briefing 
Script,  five  different  colored  flags 

e)  Testing  and  Evaluation  Using  Real  Environment: 

Read  Testing  briefing  sheet  to  participants  and  conduct  test. 
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APPENDIX  B.  SPATIAL  EXAM  FORM 
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yjpp  Directions:  Please  complete  the  above  map  to  the  best  of  your  ability.  Sketch  out  all  bulkheads  and  hatches 

k'eeT”  on  empty  space  proved  above.  Use  the  'key'  to  the  left  to  identify  objects  that  you  remember  seeing  inside 

the  environment  You  will  have  approximately  15  minutes  to  complete  this  map.  Scale  is  not  important. 
Table  Your  score  will  only  reflect  the  number  objects  placed  correctly.  This  should  be  your  own  work. 
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APPENDIX  C.  RETRIEVAL  TASK  LIST 


Task  1 :  Retrieve  Red  Flag  from  Galley 
Location:  Deep-Fat  Fryer 
Task  2:  Retrieve  Blue  Flag  from  Laundry  Room 
Location:  Steam  Cutout  Valve 
Task  3:  Retrieve  Yellow  Flag  from  Shaft  Alley 
Location:  Exposed  Shaft 
Task  4:  Retrieve  Green  Flag  from  Boiler  Space 
Location:  Fuel  Cutout  Valve 
Task  5:  Retrieve  Purple  Flag  from  Shaft  Alley 
Location:  AFFF  Hose  Reel 
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APPENDIX  D.  BRIEFING  SCRIPTS 


Introduction  Briefing  Script  (All  Groups) 

Good  Afternoon  and  welcome  to  the  Navy’s  Firefighting  School.  My  name  is  LT 
Matthew  Molmer  and  this  is  DC1  Potter.  I  am  a  student  at  the  Naval  Post  Graduate 
School  in  Monterey,  CA.  I  am  conducting  a  study  to  determine  if  using  a  PC-based 
virtual  environment  can  assist  personnel  become  familiar  with  a  new  or  unfamiliar  real 
small-scale  real  environment.  Without  any  prior  knowledge  of  the  actual  layout  of  a  real 
environment  can  personnel  become  spatially  orientate  to  a  real  space  by  first  viewing  it  as 
a  3-D  model  on  a  PC. 

This  study  does  not  test  your  ability  to  perform  in  a  real  or  virtual  environment, 
but  only  how  a  virtual  model  can  be  used  to  assist  personnel  become  familiar  with  an 
actual  environment.  The  data  collect  from  this  study  maybe  used  by  the  Navy  to  assist  in 
building  future  PC-based  gaming  software  to  enhance  training  in  the  fleet.  You  will  not 
be  perfonning  any  actual  firefighting  during  this  study. 

For  this  study  you  will  be  randomly  assigned  to  1  of  2  groups:  a  Real 
Environment  Group  or  a  Virtual  Environment  Group.  Each  of  you  will  be  asked  to  fill  out 
a  brief  pre-survey  to  gather  some  basic  demographic  infonnation  and  general  work 
experience.  Each  group  will  be  given  a  guided  tour  through  their  assigned  environments. 
After  completion  of  the  guided  tour,  each  of  you  will  be  asked  to  sketch  or  map  out  the 
layout  of  the  environment  in  which  you  toured  to  the  best  of  your  recollection.  The  sketch 
paper  will  be  provided  to  you.  When  all  personnel  have  completed  their  sketch,  your 
group  will  assemble  in  front  of  the  F-l  trainer  for  further  instruction  on  completing  a 
retrieval  task  inside  the  real  environment.  This  study  should  take  no  more  than  2  hours 
per  group  to  complete.  Upon  completion  of  each  phase  of  this  study,  you  will  be  given  a 
short  debriefing. 

If  you  have  any  questions,  at  anytime,  throughout  any  phase  of  the  study,  please 
don’t  hesitate  to  ask. 
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(HAND  OUT  CONSENT  FORMS) 

Please  read  and  sign  the  consent  form. 

(HAND  OUT  SURVEY  FORMS) 

Please  complete  the  pre-survey.  As  outlined  in  the  consent  form,  your  answers 
will  be  held  strictly  confidential  and  will  only  be  used  in  conducting  this  study.  Please  be 
sure  to  write  your  ID  number  in  the  top  right  hand  comer  of  your  survey  form. 

(PASS  AROUND  RANDOM  NUMBER  ASSIGNMENT  BIN) 

Please  take  a  number.  This  will  be  your  group  assignment  for  reminder  of  this 
study.  Please  write  that  number  in  the  top  right-hand  corner  of  your  survey  form.  Group  1 
is  designated  as  the  Real  Environment  Group.  Group  2  is  designated  as  the  Virtual 
Environment  Group. 

(COLLECT  ALL  FORMS) 

(DISMISS  VE  GROUP  TO  RECONVENE) 

VE  Group,  please  assemble  back  in  this  classroom  in  1  hour. 

(ASSEMBLE  REAL  ENVIRONMENT  GROUP  IN  FRONT  OF  F-l  TRAINER) 
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Guided  Tour  (All  Groups) 

During  the  guided  tour  of  the  F-l  trainer  you  will  explore  the  spaces  on  the  first 
floor  of  the  F-l  trainer.  In  each  space  your  attention  will  be  directed  to  specific  aspects 
and  equipment  within  that  space.  Your  time  in  each  space  will  be  approximately  5 
minutes.  You  will  not  be  able  to  take  written  notes  or  draw  sketches  of  the  spaces. 

After  your  exploration  of  the  all  the  spaces  is  completed,  your  group  will 
reconvene  in  the  classroom  and  each  of  you  will  be  instructed  to  recreate  the  spatial 
layout  with  a  written  sketch  on  a  form  that  I  will  provide  to  you. 

You  may  ask  safety  and  technical  questions  anytime  during  the  guided  tour  phase 
of  the  study.  You  are  asked  not  to  ask  question  relating  to  the  spatial  layout  of  the  spaces. 
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Experiment  Briefing  (All  Groups) 

(MEET  PARTICIPANTS  IN  FRONT  OF  F-l  TRAINER) 

You  will  be  performing  two  randomly  selected  retrieval  tasks  in  the  F-l  trainer. 
Each  task  consists  of  retrieving  a  specific  colored  flag  from  one  of  the  spaces  you 
explored  during  the  guided  tour  phase.  I  will  give  you  the  specific  flag  color,  the  space, 
and  location  of  the  flag.  Your  instructions  are  to  retrieve  the  flag  as  quickly  as  possible 
and  you  time  will  be  recorded.  Your  group  will  be  brought  to  the  starting  point  from  the 
2nd  deck  so  you  do  not  traverse  through  the  1st  deck  testing  area.  I  ask  that  you  not  run 
during  your  time  in  the  trainer.  You  may  proceed  however  at  a  hurried  pace  to  retrieve 
your  flag.  After  you  have  retrieved  your  2nd  flag,  please  wait  off  to  the  side  until  your 
entire  group  has  finished  their  task  and  you  will  all  exit  the  F-l  trainer  together. 

(WALK  GROUP  UP  TO  2nd  DECK  AND  DOWN  TO  STARTING  POINT  ON 
1st  DECK) 

(GROUP  RETRIEVAL  TASK  INSTRUCTIONS) 

Task  1:  Retrieve  Red  Flag  from  Galley 
Location:  Deep-Fat  Fryer 
Task  2:  Retrieve  Blue  Flag  from  Laundry  Room 
Location:  Steam  Cutout  Valve 
Task  3:  Retrieve  Yellow  Flag  from  Shaft  Alley 
Location:  Exposed  Shaft 
Task  4:  Retrieve  Green  Flag  from  Boiler  Space 
Location:  Fuel  Cutout  Valve 
Task  5:  Retrieve  Purple  Flag  from  Shaft  Alley 
Location:  AFFF  Hose  Reel 


42 


APPENDIX  E.  PARTICIPANT  CONSENT  FORM 


Naval  Postgraduate  School 

Participant  Consent  Form  & 
Minimal  Risk  Statement 


Introduction.  You  are  invited  to  participate  in  a  study  entitled  Spatial  Orientation 
in  small  scale  Virtual  Environments  being  conducted  by  the  Naval  Postgraduate  School 
MOVES  Institute. 

Procedures.  If  I  agree  to  participate  in  this  study,  I  understand  I  will  be  provided 
with  an  explanation  of  the  purposes  of  the  research,  a  description  of  the  procedures  to  be 
used,  identification  of  any  experimental  procedures,  and  the  expected  duration  of  my 
participation.  Synopsis :  There  will  be  three  phases  of  the  experiment:  (1)  a  30  minute 
environment  familiarization  phase  (2)  an  execution  phase  lasting  approximately  1  hour, 
during  which  you  will  be  expected  to  accomplish  a  number  of  tasks  related  to  navigating  in 
a  real  building  environment  (3)  and  an  evaluation  phase,  during  which  you  will  expected  to 
take  a  short  written  exam. 

Risks  and  Benefits.  I  understand  that  this  project  does  not  involve  greater  than 
minimal  risk  and  involves  no  known  reasonably  foreseeable  risks  or  hazards  greater  than 
those  encountered  in  everyday  life.  I  have  also  been  informed  of  any  benefits  to  myself  or 
to  others  that  may  reasonably  be  expected  as  a  result  of  this  research. 

Compensation.  I  understand  that  no  tangible  reward  will  be  given.  I  understand 
that  a  copy  of  the  research  results  will  be  available  at  the  conclusion  of  the  experiment. 

Confidentiality  &  Privacy  Act.  I  understand  that  all  records  of  this  study  will  be 
kept  confidential  and  that  my  privacy  will  be  safeguarded.  No  information  will  be  publicly 
accessible  which  could  identify  me  as  a  participant,  and  I  will  be  identified  only  as  a  code 
number  on  all  research  forms.  I  understand  that  records  of  my  participation  will  be 
maintained  by  NPS  for  five  years,  after  which  they  will  be  destroyed. 

Voluntary  Nature  of  the  Study.  I  understand  that  my  participation  is  strictly 
voluntary,  and  if  I  agree  to  participate,  I  am  free  to  withdraw  at  any  time  without  prejudice. 

Points  of  Contact.  I  understand  that  if  I  have  any  questions  or  comments  regarding 
this  project  upon  the  completion  of  my  participation,  I  should  contact  the  Principal 
Investigator,  Perry  McDowell,  656-7591,  mcdowell@nps.edu.  Any  medical  questions 
should  be  addressed  to  LTC  Eric  Morgan,  MC,  USA,  (CO,  POM  Medical  Clinic),  (831) 
242-7550,  eric.morgan@nw.amedd.army.mil. 
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Statement  of  Consent.  I  have  read  and  understand  the  above  infonnation.  I  have 
asked  all  questions  and  have  had  my  questions  answered.  I  agree  to  participate  in  this 
study.  I  will  be  provided  with  a  copy  of  this  form  for  my  records. 

Participant’s  Signature  Date 

Researcher’s  Signature  Date 
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APPENDIX  F.  PARTICIPANT  SURVEY 
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Simulations  Experience 
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